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Abstract 

We survey results on the hardness of approximating combinatorial optimization problems. 
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1 Introduction 

Hundreds of interesting and important combinatorial optimization problems are NP-hard, and so it 
is unlikely that any of them can be solved by an efficient exact algorithm. Short of proving P = NP, 
when one deals with an NP-hard problem one can either hope to design an exact algorithm that runs 
in polynomial time on "many" instances but has exponential worst-case running time, or to design 
an efficient algorithm that finds sub-optimal solutions. In this paper we focus on approximation 
algorithms, that are algorithms of the second kind with a provably good worst-case ratio between 
the value of the solution found by the algorithm and the true optimum. 

Formally, we say that an algorithm is r-approximate for a minimization problem (respectively, 
a maximization problem) if, on every input, the algorithm finds a solution whose cost is at most 
r times the optimum (respectively, at least 1/r times the optimum). The ratio r, which is always 
> 1, is also called the performance ratio of the algorithm. 

For some problems, it is possible to prove that even the design of an r-approximate algorithm 
with small r is impossible, unless P = NP. Results of this kind, called inapproximability results, 
are the subject of this survey. 



1.1 A Brief Historical Overview 

The seeming intractability of many combinatorial optimization problems was observed already in 
the 1960s, motivating the development of suboptimal heuristic algorithms and, in particular, the 
notion of approximation algorithm as defined above. An early example of analysis of an approxi- 
mation algorithm is a paper of Graham on scheduling problems Gra63 . 

The theory of NP-completeness |Coo7H ILev73| IKar72j , and its success in classifying the com- 
plexity of optimization problems, provided added motivation to the study of efficient suboptimal 
algorithms and to the rigourus analysis of approximation algorithms. A 1973 seminal paper by 
Johnson jJoh74j gave the field its current foundations. Johnson considers the problems Max SAT, 
Set Cover, Independent Set, and Coloring.^ He notes that there is a 2-approximate algorithm for 
Max SAT, and that this can be improved to 8/7 for Max E3SAT, the restriction of the problem 
to instances in which every clause contains exactly three literals. He also presents a (1 + Ink)- 
approximate algorithm for Set Cover, where k is the size of the largest set in the collection, and he 
notes that natural heuristics for Coloring and Independent Set fail to give even a n^~^ approxima- 
tion for e > 0. 

^The Max SAT problem is defined in Section 13.11 the Independent Set problem is defined in Section 14.21 tfie 
Coloring problem is defined in Section 15!^ and the Set Cover problem is defined in Section 1^31 
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Thirty years after Johnson's paper, the design and analysis of approximation algorithms has 
become a large and successful research area. A book edited by Hochbaum |Hoc96j and two more 
recent textbook s |ACG+99l IVaznij give an extensive overview of the field. The book by Ausiello 



et al. lACG+QQj also includes a list of known results for hundreds of problems. 

As the field progressed and matured, it became apparent that different combinatorial optimiza- 
tion problems behaved different from the point of view of approximability. For some problems, 
researchers were able to devise polynomial time approximation schemes (abbreviated PTAS), that 
is, to devise an r-approximate algorithm for every r > 1. For other problems, such as Max SAT 
and Vertex Cover, ^ r-approximate algorithms for constant r were known, but no approximation 
scheme. Other problems like Set Cover had no known constant factor approximation algorithm, 
but were known to admit an approximation algorithm with a performance ratio that was slowly 
growing (logarithmic) in the input length. Finally, there were problems like Independent Set and 
Coloring for which not even a slowly growing performance ratio was known to be achievable in poly- 
nomial time. On the other hand, until 1990, there was no negative result showing that the existence 
of approximation schemes for any of the above problems would imply P = NP or other unlikely 
consequences. Indeed, few inapproximability results were known at all. Furthermore, there was a 
general intuition that, in order to prove inapproximability for the above mentioned problems (and 
others), it was necessary to depart from the standard techniques used to prove the NP-hardness of 
problems. 

To see the limitations of standard NP-hardness proofs, consider for example the proof that 
solving optimally the Independent Set problem is NP-hard. The proof starts from a 3SAT instance 
</? and constructs a graph G and an integer k such that if (p is satisfiable then G has an independent 
set of size k, and if (p is not satisfiable then all independent sets in G have size at most k — 1. 
It then follows that a polynomial time exact algorithm for solving Independent Set, together with 
the reduction, would imply a polynomial time algorithm for 3SAT, and, from Cook's theorem, we 
would have a polynomial time algorithm for every problem in NP. Consider now the question of 
the approximability of the Independent Set problem. Looking at the reduction from 3SAT (that we 
describe in Section f4.2j) we can see that if p has an assignment that satisfies all the clauses except 
c ones, then G has an independent set of size k — c and, given such an assignment, the independent 
set is easy to construct. Furthermore, the instances of 3SAT produced by Cook's theorem are such 
that it is easy to find assignments that satisfy all the clauses but one. In conclusion, the instances 
that we get by reducing a generic NP problem to Independent Set are very easy to approximate. 

To derive an inapproximability result, we need a much stronger reduction. Suppose we want to 
prove that no 2-approximate algorithm exists for the Independent Set problem assuming P ^ NP. 
Then a natural proof strategy would be to start from an NP-complete problem, say, 3SAT, and 
then reduce an instance of 3SAT to an instance of Independent Set, with the property that, 
for some k, if p is satisfiable then 

O-PTindcpcndcnt Set(G^) > k (1) 

and if ip is not satisfiable then 

OPTindependent Set{G^) < k/2 . (2) 

Suppose we have such a reduction, and suppose we also have a 2-approximate algorithm for Inde- 
pendent Set; then the algorithm, given G^p, will find a solution of cost at least k/2 if and only if (p 

^We define the Vertex Cover problem in Section f4.2l 
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is satisfiable. The approximation algorithm then gives a polynomial time algorithm for 3SAT, and 
so no such algorithm can exist if P 7^ NP. 

All the NP-completeness proofs for graph problems before 1990, however, can be essentially 
described as follows: we start from the computation of a generic non-deterministic Turing machine, 
then we encode its computation as a 3SAT formula, using the construction of Cook's theorem, and 
then we reduce 3SAT to the problem of interest (the reduction may be presented as a sequence 
of reductions involving several intermediate problems, but it can always be thought of as a direct 
reduction from 3SAT) by encoding variables and clauses of the formula as sub-graphs connected 
in a proper way. The computation of a Turing machine is very sensitive to small changes, and it 
seems impossible to generate an inapproximability gap starting from a fragile model and applying 
"local" reductions. The only inapproximability results that can be proved with such reductions 
are for problems that remain NP-hard even restricted to instances where the optimum is a small 
constant. For example, in the Metric Min fc-Center problem it is NP-hard to decide whether the 
optimum has cost 1 or 2, and so no algorithm can have a performance ratio smaller than 2 unless 
P = NP |HS85j . Similarly, in the Coloring problem it is NP-hard to decide wether the optimum 
has cost 3 or 4, and so no algorithm has performance ratio smaller than 4/3 unless P = NP, and 
Carey and Johnson |(T.T76j show that the gap can be "amplified" to k versus 2A; — 4 for constant 
k, ruling out also algorithms with performance ratio smaller than 2. Most interesting problems, 
however, become trivial when restricted to inputs where the optimum is a constant. 

To prove more general inapproximability results it seemed necessary to first find a machine 
model for NP in which accepting computations would be "very far" from rejecting computations. 
Before such a model was discovered, an important piece of work on inapproximability was due to 
Papadimitriou and Yannakakis, who showed that, assuming that Max 3SAT does not have a PTAS, 
then several other problems do not have a PTAS |iPY91j . Berman and Schnitger BS92_ proved 
that if Max 2SAT does not have a PTAS then, for some c > 0, the Independent Set problem cannot 
be approximated within a factor n*^. 

The modern study of inapproximability was made possible by the discovery, due to Feige et 



al. jFCL^96j in 1990, that probabilistic proof systems could give a robust model for NP that could 
be used to prove an inapproximability result for the Independent Set problem.^ A year later, Arora 
et al. IALM+98| proved the PCP Theorem, a very strong characterization of NP in terms 

of proof systems, and showed how to use the PCP Theorem to prove that Max 3SAT does not 
have a PTAS. Using the reductions of [PY9T] (and others [PV93. ■BP89] V the PCP Theorem gave 
inapproximability results for several other problems. 

An explosion of applications to other problems soon followed, such as the work of Lund and 
Yannakakis |LY94j on Set Cover and Coloring and the work of Arora et al. |ABSS97j on lattice 
problems, that appeared in 1993. Later work focused on stronger inapproximability results, ob- 
tained both by proving stronger versions of the PCP Theorem and by devising better reductions. 
Despite great technical difficulties, very rapid progress was made in the mid 1990s. It is interesting 
to read a survey paper by Bellare |Be]9fij . written in the middle of the most exciting period of that 
time. Some of the open questions raised in the paper were solved a few months later. We now 
know inapproximability results for Max 3SAT |Hasnij . Set Cover |Fei98j . Independent Set |Has99j 
and Coloring |FK98j showing that the performance ratios of the algorithms presented in Johnson's 
paper |Joh74j are best possible (assuming P 7^ NP or similar complexity assumptions). 

■^Another, less influential, connection between probabilistic proof checking and inapproximability was discovered 
around the same time by Condon |Uon93| . 
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Despite all these achievements, the study of inapproximability results is still a vibrant field, 
with radically new insights and major technical achievements emerging at a steady pace. 

1.2 Organization of This Survey 

To give justice to the breadth and the diversity of this field, we chose to present it from different 
perspectives. 

After a review of technical definitions (Section I^J, we disucss the PCP Theorem in Section |3J 
inapproximability results that follow from the PCP Theorem in Section ^ and stronger results 
that follow from "optimized" versions of the PCP theorem and/or from "optimized" reductions in 
Sectional This is a "textbook" presentation of inapproximability results, similar, for example, to 
the one in IVaznil Chapter 29]. We include some proofs and we discuss problems for which the 
reductions are relatively simple and yield inapproximability results that are tight or almost tight. 
The focus of Sections El IH and|Slis on techniques used to prove inapproximability results. 

In Section El we review what is known for various fundamental problems. This section has 
mostly a reference character, and the focus is on problems. We give particular space to problems 
for which there are important open questions and that the reader could use as research topics. 

After approaching the field from the perspectives of techniques and of results for specific prob- 
lems, we discuss in Section |7| a number of alternate questions that have been pursued. One ques- 
tion relates to the fact that the best possible performance ratios for natural problems that do not 
have a PTAS were observed to often be 0(1), O(logn) or n'^^^\ A survey paper by Arora and 
Lund IAL96' emphasizes this regularity. For a subclass of optimization problems it was actually 
proved ^re95, KSTWOO that only a finite number of approximability behaviour were possible. 
However, we now know of a natural problem for which the best possible performance ratio is 
0(log*n), of another for which it is somewhere between 0((logn)^) and 0((logn)^, and another 
for which it is somewhere between O(loglogn) and O (log n/ log log n). Contrary to previous intu- 
ition, it now appears that it is not possible to classify the approximability of natural problems into 
a small number of simple cases. Another topic that we discuss is the relation between instances 
that are "hard" for linear and semidefinite relaxation and instances that are generated by approx- 
imation preserving reductions. We also discuss the study of complexity classes of combinatorial 
optimization problems, of relations between average-case complexity and inapproximability, and of 
the issue of witness length in PCP constructions. 

We make some concluding remarks and we discuss a small sample of open questions in section |H1 

1.3 Further Reading 

Several excellent surveys on inapproximability results have been written. 

A paper by Arora and Lund AL96 gives a very good overview of the techniques used to 
prove inapproximability results. Our discussion in Section 17.41 is motivated by the classification 
scheme defined in |AL96j . Arora has written two more survey papers |Aro98a[ IAro98a] on PCP 
and inapproximability. 

The subject of strong inapproximability results is discussed in a paper by Bellare |Bel96j . that 
we have already mentioned, and in a more recent paper by Feige |Feif)2aj . 

A book by Ausiello and others [ACG"'"99] has an extensive compendium of known inapproxima- 
bility results for hundreds of problems^ and chapters devoted to PCP, inapproximability results, 

''At the time of writing, the compendium is available at|http://www.nada. kth.se/~viggo/problemlist/compendium. html 
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and complexity classes of optimization problems. The books by Papadimitriou |Pap94 and by 
Vazirani IVazOlj have chapters devoted to inapproximability results. 



2 Some Technical Preliminaries 

We assume the reader is familiar with the basic notions of algorithm, running time and big-Oh no- 
tation. Such background can be found, for example, in the introductory chapter of Papadimitriou's 
textbook |Pap94| . 



NP-Completeness 

We assume that all combinatorial objects that we refer to (graphs, boolean formulas, families of 
sets) are represented as binary strings. For a binary string x, we denote its length as |x|. We 
represent a decision problem as a language, that is, as the set of all inputs for which the answer 
is YES. We define P as the class of languages that can be decided in polynomial time. We define 
NP as the class of languages L such that there is a polynomial time computable predicate V and 
a polynomial q{) such that x £ L if and only if there is w, \w\ < q{\x\) such that V^XjW) accepts. 
We think of as a proof, or witness that x is in the language. 

For two languages Li and L2, we say that Li reduces to L2, and we write Li <m L2 if there is 
polynomial time computable / such that x £ Li ii and only if /(x) G L2. A language A is NP-hard 
if every language L in NP reduces to A. A language is NP-complete if it is NP-hard and it belongs 
to NP. 



NPO Problems 

A combinatorial optimization problem O is defined^ by a cost function costoO that given an instance 
X of the problem and a solution s outputs costo(x,s) which is either the cost of the solution (a 
non-negative real number) or the special value _L if s is not a feasible solution for x. For every x, 
there is only a finite number of feasible solutions s such that costo(x, s) 7^ -L. If O is a maximization 
problem (respectively, maximization), then our goal is, given x to find a feasible solution s such that 
cost(x,s) is smallest (respectively, largest). We denote by opto(x) the cost of an optimal solution 
for X. 

For example, in the independent set problem, an instance is an undirected graph G = {V,E), a 
feasible solution is a subset 5 C 1/ such that no two vertices of S are connected by an edge. The 
cost of a feasible S is the number of vertices in S. 

An optimization problem O is an NP-optimization problem, and it belongs to the class NPO, if 
costoO is computable in polynomial time and if there is a polynomial q such that for every instance 
and every solution s that is feasible for x we have \s\ < q{\x\). The independent set problem is 
clearly an NPO problem. 

If P = NP, then for every NPO problem there is a polynomial time optimal algorithm. If 
P 7^ NP, then none of the NPO problems considered in this paper has a polynomial time optimal 
algorithm. 

^Other conventions are possible, which are typically equivalent to the one we describe here. 
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Approximation 

If O is an NPO minimization problem, r > 1, and A is an algorithm, we say that A is an 
r-approximate algorithm for O if, for every instance x, A{x) is a feasible solution for x and 
costoix, A{x)) < r ■ optQ(x). If O is a maximization problem and r > 1, then wc say that A 
is an r-approximate algorithm if costo(a;, ^(x)) > optQ(x)/r. Another common convention, that 
we do not use here, is to say that an algorithm is r-approximate for a maximization NPO problem 
O, with r < 1 if, for every x, costo(x, ^(x)) > r • opto(x). Finally, sometimes we let r = r(n) be a 
function of the length of the instance x, or of some other parameter related to the size of x. 

For example, a 2-approximate algorithm for the independent set problem would be an algorithm 
that given a graph G outputs a feasible independent set S such that l^l > |-S'*|/2, where S* is an 
optimal independent set. As we will se later, such an approximation algorithm can exist if and only 
if P = NP. 

Complexity Classes 

We have already defined the classes NP and P. We also define, for later reference, classes of problems 
solvable by efficient probabilistic algorithms. We define BPP as the class of decision problems 
(languages) that can be solved in polynomial time by a probabilistic algorithm that, on every 
input, has a probability of error at most 1/3. We also define two subclasses of BPP. The class RP 
contains decision problems that admit a polynomial time probabilistic algorithm that is correct 
with probability one if the correct answer is "NO," and is correct with probability at least 1/2 if 
the correct answer is "YES." The class ZPP contains decision problems that admit a probabilistic 
algorithm that, on every input, runs in average polynomial time and always returns the correct 
answer. (The average is taken only over the random choices of the algorithm.) 

We say that an algorithm runs in quasi-polynomial time if, for some constant c, it runs in 
time 0(n(^°s")'') on inputs of length n. We denote by QP the class of decision problems solvable 
by deterministic quasi-polynomial time algorithms, and by QRP the relaxation of RP to quasi- 
polynomial running time. 

It is considered extremely unlikely that NP be contained in any of the complexity classes defined 
above. Furthermore, if, say NP C RP, then for every NPO problem there is a quasi-polynomial time 
probabilistic algorithm that on every input outputs, with high probability, an optimal solution. 

3 Probabilistically Checkable Proofs 

As discussed in the introduction, probabilistically checkable proofs (PCPs) provide a "robust" 
characterization of the class NP. When we reduce a generic NP problem L to 3SAT using Cook's 
theorem, we give a way to transform an instance x into a 3CNF formula (fx so that if x G L then 
tpx is satisfiable, and if x L then ip^ is not satisfiable. Following the proof of Cook's theorem, 
however, we see that it is always easy (even when x ^ L) to construct an assignment that satisfies 
all the clauses of ipx except one. 

Using the PCP Theorem one can prove a stronger version of Cook's theorem, that states that, 
in the above reduction, if x G -L then (fx is satisfiable, and if x ^ L then there is no assignment 
that satisfies even a 1 — e fraction of clauses of fx, where e > is a fixed constant that does not 
depend on x. This immediately implies that Max 3SAT does not have a PTAS (unless P = NP), 
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and that several other problems do not have a PTAS either (unless P = NP), using the reductions 
of Papadimitrious and Yannakakis IPYOT' and others. 

We define PCPs by considering a probabilistic modification of the definition of NP. We consider 
probabilistic polynomial time verifiers V that are given an input x and "oracle access" to a witness 
string w. We model the fact that y is a probabilistic algorithm by assuming that V, besides the 
input X and the witness w, takes an additional input R, that is a sequence of random bits. Then 
V performs a deterministic computation based on x, w and R. For fixed w and x, when we say 
"V^{x) accepts" we mean "the event that V accepts when given oracle access to witness w, input 
x, and a uniformly distributed random input R." When we refer to the "probability that V'^{x) 
accepts," we take the probability over the choices of R. 

We say that a verifier is (r(n), g(n))-restricted if, for every input x of length n and for every w, 
V'^{x) makes at most q{n) queries into w and uses at most r(n) random bits. 

We define the class PCP[r(n), q{n)] as follows. A language L is in PCP[r(n), q{n)] if there is an 
(r(n), g(n))-restricted verifier V such that if x E L, then there is w such that V'^{x) accepts with 
probability 1 and if x L then for every w the probability that y"'(x) accepts is < 1/2. 

We also consider the following more refined notation. We say that a language L is in 
PCPc(„)^5(„) [r(n), g(n)] if there is an (r(n), g(n))-restricted verifier V such that if x G L, then 
there is w such that V'^{x) accepts with probability at least c(n), and if x L then for every w 
the probability that ^"(x) accepts is at most s(n). Of course, the definition makes sense only if 
< s{n) < c(n) < 1 for every n. The parameter c(n) is called the completeness of the verifier and 
the parameter s(n) is called the soundness error, or simply the soundness of the verifier. 

Note that if r(n) = O(logn) then the proof-checking can be derandomized, that is, V can 
be simulated by a polynomial time deterministic verifier that simulates the computation of V on 
each of the 2^^"'^ = n*^*^^-* possible random inputs and then computes the probability that ^"(x) 
accepts, and then accepts if and only if this probability is one. It then follows that, for example, 
PCP[0(log n), 0(log n)] C NP. The PCP Theorem shows a surprising converse. 

Theorem 1 (PCP Theorem) NP = PCP[0(logn), 0(1)]. 

The theorem was proved in |AS98llALM"'"98j . motivated by a relation between PCP and approx- 
imation discovered in _ FGL"'"96| . The actual proof relies on previous work, as well as on several new 
results. A survey paper by David Johnson |.Toh92j and the introductions of the theses of Madhu 
Sudan |Sud92j and Sanjeev Arora |Aro94j give an overview of the line of work that led to the PCP 
Theorem. 

3.1 PCP and the Approximability of Max SAT 

In the Max 3SAT problem we are given a 3CNF boolean formula, that is, a boolean formula in 
conjunctive normal form (AND-of-OR of literals, where a literal is either a variable or the negation 
of a variable) such that each term in the conjunction is the OR of at most three literals. The goal 
is to find an assignment that satisfies the largest possible number of terms. 

Theorem 2 The PCP Theorem implies that there is an ei > such that there is no polynomial 
time (1 + £i)- approximate algorithm for MAX-3SAT, unless P = NP. 

Proof: Let L S PCP[0(logn), be an NP-complete problem, where q is a. constant, and let V 
be the (0(log n), (7)-restricted verifier for L. We describe a reduction from L to Max 3SAT. Given 
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an instance x of L, our plan is to construct a 3CNF formula (fx with m clauses such that, for some 
ei > to be determined, 

X £ L =^ V'x is satisfiable , . 

X ^ L =^ no assignment satisfies more than (1 — ei)m clauses of ipx- 

Once (jSJ is proved, the theorem follows. 

We enumerate all random inputs R for V. The length of each string is = 0(log |x|), so 

the number of such strings is polynomial in For each R, V chooses q positions i^, • • • and 
a Boolean function fji : {0, 1}^ {0, 1} and accepts iff fR{'w^R, . . . , w^r) = 1. 

We want to simulate the possible computation of the verifier (for different random inputs R 
and different witnesses w) as a Boolean formula. We introduce Boolean variables xi,. . . ,xi, where 
I is the length of the witness w. 

For every R we add clauses that represent the constraint . . . jX^n) = 1. This can be 

done with a CNF of size < 2^, that is, we would need to add at most 2^ clauses if we were allowed 
to write a gCNF expression. But we also need to "convert" clauses of length q to clauses length 3, 
which can be done by introducing additional variables, as in the standard reduction from /cSAT to 
3SAT (for example X2 V xio V xu V xi2 becomes (x2 V xiq V ur) A {ijr V xu V X12)). Overall, this 
transformation creates a formula (fx with at most q2'^ 3CNF clauses. 

Let us now see the relation between the optimum of <pz as an instance of Max 3SAT and the 
question of whether z € L. 

If z G L, then there is a witness w such that V'^{z) accepts for every R. Set Xi = Wi, and set 
the auxiliary variables appropriately, then the assignment satisfies all clauses, and ipz is satisfiable. 

If z ^ L, then consider an arbitrary assignment to the variables Xi and to the auxiliary variables, 
and consider the string w where Wi is set equal to Xj. The witness w makes the verifier reject for 
half of the R G {0, Ij^d^D, and for each such R, one of the clauses representing //j fails. Overall, 
at least a fraction ei = of clauses fails. □ 

Interestingly, the converse also holds: any gap-creating reduction from an NP-complete problem 
to Max 3SAT implies that the POP Theorem must be true. 

Theorem 3 If there is a reduction as in ^ for some problem L in NP, then L G 
PCP[0(logn), 0(1)]. In particular, if L is NF-complete then the PGP Theorem holds. 

Proof: We describe how to construct a verifier for L. V on input z expects to be a satisfying 
assignment for ifz- V picks O(^) clauses of ipz at random, and checks that w satisfies all of them. 
The number of random bits used by the verifier is 0(^ logm) = 0(log \z\). The number of bits of 
the witness that are read by the verifier is 0{j^) = 0(1). 

z £ L =^ is satisfiable 

^ 3w such that y"(z) always accept. 



z ^ L ^ \/w a fraction ei of clauses of ip are unsatisfied by w 
yw V^{z) rejects with probability > | 



□ 
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4 Basic Reductions 



We have seen that the PCP Theorem is equivalent to the inapproximabihty of Max 3SAT and 
other constraint satisfaction problems. In this section we will see several reductions that prove 
inapproximabihty results for other problems. 

4.1 Max 3SAT with Bounded Occurrences 

We begin with a reduction from the Max E3SAT problem on general instances to the restriction 
of Max E3SAT to instances in which every variable occurrs only in a bounded number of clauses. 
The latter problem will be a useful starting point for other reductions. 
For the reduction we will need expander graphs of the following type. 

Definition 4 (Expander Graph) An undirected graph G = {V,E) is a 1-expander if, for every 
subset S QV , \S\ < \V\/2, the number of edges e{S,V — S) having one endpoint in S and one in 
V — S is at least \S\ . 

For our purposes, it will be acceptable for the expander graph to have multiple edges. It is easy 
to prove the existence of constant-degree 1-expanders using the probabilistic method. Polynomial- 
time constructible 1-expanders of constant degree can be derived from |GG81j . and, with a smaller 
degree, from |LPS88j . Let d be a constant for which degree-d 1-expanders can be constructed in 
polynomial time, {d = 14 suffices using the construction of |LPS88 I.') 

Let now ip be an instance of 3SAT with n variables xi, . . . ,Xn and m clauses. For each variable 
Xi, let occj be the number of occurrences of Xi, that is, the number of clauses that involve the 
literal Xi or the literal Xi. We write Xi £ Cj if the variable Xi occurs in clause Cj. Notice that 
Y17=i ^^^i ~ -^o^ each construct a 1-expander graph Gi = {Vi^Ei) where Vi has occj vertices, 
one for each occurrence of Xi in (p. We denote the vertices of Vi as pairs such that Xi occurrs 
in Cj . Each of these graphs has constant degree d. 

We define a new instance V of Max E3SAT with A'' = 3m variables Y = {yi,j}i^[n],Xi&Cj^ 
for each occurrence of each variable in ip. For each clause of 99 we put an equivalent clause in tp. 
That is, if Cj = {xa V x;, V Xc) is a clause in ip, then (yaj V yb,j V ycj) is a clause in We call these 
clauses the primary clauses of if:. Note that each variable of ij^ occurs only in one primary clause. 

To complete the construction of for every variable xi in ip, and for every edge ([i, j], 
in the graph Gi, we add the clauses (y^j- V y^'j) and (y^j- V y^'j) to ijj. We call these clauses the 
consistency clauses of Notice that if yjj = yj/ j then both consistency clauses are satisfied, while 
if yjj- / yj'j- then one of the two consistency clauses is contradicted. 

This completes the construction of ip. By construction, every variable occurrs in at most 2d+\ 
clauses of ■0i and ■0 has M = m + 3dm clauses. 

We now claim that the cost of an optimum solution in ip is determined by the cost of an optimum 
solution in (p and, furthermore, that a good approximation algorithm applied to ip returns a good 
approximation for (p. We prove the claim in two steps. 

Claim 5 If there is an assignment for ip that satisfies m — k clauses, then there is an assignment 
for tp that satisfies > M — k clauses. 

Proof: This part of the proof is simple: take the assignment for ip and then for every variable y^j- 
of Ip give to it the value that the assignment gives to Xi . This assignment satisfies all the consistency 
clauses and all but k of the remaining clauses. □ 
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Claim 6 If there is an assignment for -0 that leaves k clauses not satisfied, then there is an as- 
signment for if that leaves < k clauses not satisfied. 

Proof: This is the interesting part of the proof. Let Ojj be the value assigned to yij. We first 
"round" the assignment so that all the consistency clauses are satisfied. This is done by defining 
an assignment bi, where, for every i, the value bi is taken to be the majority value of aij over all 
j such that Xi G Cj, and we assign the value bi to all the variables yij. The assignment bi satisfies 
all the consistency clauses, but it is possible that it contradicts some primary clauses that were 
satisfied by aij. We claim that, overall, the bi assignment satisfies at least as many clauses as the 
gnment. Indeed, for each i, if bi differs from the ajj for, say, t values of j, then there can 
be at most t primary clauses that were satisfied by Uij but are contradicted by bi. On the other 
hand, because of the consistency clauses being laid out as the edges of a 1-expander graph, at least 
t consistency clauses are contradicted by the gnment for that value of i alone, and so, the 

bi assignment can be no worse. 

We conclude that bi assignment contradicts no more clauses of if) than are contradicted by aj^-, 
that is, no more than k clauses. When we apply bi as an assignment for 99, we see that bi contradicts 
at most k clauses of 99. □ 

In conclusion: 

• If 99 is satisfiable then ip is satisfiable; 

• If every assignment contradicts at least an e fraction of the clauses of then every assignment 
contradicts at least an + 3(i) fraction of the clauses of if). 

Theorem 7 There are constants d and £2 and a polynomial time computable reduction from 3SAT 
to Max 3SAT-d such that if ip is satisfiable then f{(p) is satisfiable, and if ip is not satisfiable then 
the optimum of f{ip) is less than 1 — £2 times the number of clauses. In particular, if there is an 
approximation algorithm for Max 3SAT-d with performance ratio better than (1 — £2); then P = NP. 

4.2 Vertex Cover and Independent Set 

In an undirected graph G = {V, E) a vertex cover is a set C C 1/ such that for every edge (n, v) € E 
we have either u £ C or v G C, possibly both. An independent set is a set S O V such that for 
every two vertices u,v £ S we have {u, v) E. It is easy to see that a set C is a vertex cover in G 
if and only if — C is an independent set. It then follows that the problem of finding a minimum 
size vertex cover is the same as the problem of finding a maximum size independent set. From the 
point of view of approximation, however, the two problems are not equivalent: the Vertex Cover 
problem has a 2-approximate algorithm (but, as we see below, it has no PTAS unless P = NP), 
while the Independent Set problem has no constant-factor approximation unless P = NP. 

We give a reduction from Max ESS AT to Independent Set. The reduction will also prove 
intractability of Vertex Cover. If we start from an instance of Max E3SAT-(i we will get a bounded 
degree graph, but the reduction works in any case. The reduction appeared in |PY91j . and it is 
similar to the original proof of NP-completeness of Vertex Cover and Independent Set jKar72j . 

Starting from an instance (p of ESS AT with n variables and m clauses, we constuct a graph 
with 3m vertices; the graph has a vertex Vij for every occurrence of a variable Xi in a clause Cj. 
For each clause Cj, the three vertices corresponding to the three literals in the clause are joined by 
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Figure 1: Graph construction corresponding to the 3CNF formula 99 = (xi V X2 V X3) A 

(x2 V X3 V 2:5) A {xi V X4 V X5) A {xi V X3 V X5). 

edges, and form a triangle (we call such edges clause edges). Furthermore, if a variable Xi occurrs 
positively in a clause Cj and negated in a clause Cji, then there is an edge between the vertices 
Vij and Vi^ji (we call such edges consistency edges). Let us call this graph G^p. See Figure ^ for an 
example of this construction. 

Note that if every variable occurrs in at most d clauses then the graph has degree at most d + 2. 

Claim 8 There is an independent set of size > t in if and only if there is an assignment that 
satisfies > t clauses in if. 

Proof: Suppose we have an assignment Oj that satisfies t clauses. For each clause Cj, let us pick a 
vertex Vij that corresponds to a literal of Cj satisfied by Oj. We claim that the set of picked vertices 
is an independent set in G^. To prove the claim, we note that we picked at most one vertex from 
each triangle, so that we do not violate any clause edge, and we picked vertices consistent with the 
assignment, so that we could not violate any consistency edge. 

For the other direction, suppose we have an independent set with t vertices. The vertices must 
come from t different triangles, corresponding to t different clauses. We claim that we can satisfy 
all such clauses. We do so by setting an assingment so that Xi takes a value consistent with the 
vertices Vij in the independent set, if any. Since consistency edges cannot be violated, this is a well 
defined assignment, and it satisfies t clauses. □ 

If we combine this reduction with Theorem [3 we get the following result. 

Theorem 9 There is a polynomial time computable function mapping instances (p of 3SAT into 
graphs G^ of maximum degree d + 2 such that if is satisfiable then G^p has an independent set of 
size at least N/3 (and a vertex over of size at most 2N/3, where N is the number of vertices, and 
if if is not satisfiable then every independent set in G^ has size at most N ■ {1 — £2)/3, and every 
vertex cover has size at least N ■ (2 + £2)/3. In particular, if there is an approximation algorithm 
for Independent Set in degree-{d + 2) graphs with performance ratio better than 1/(1 — £2), or if 
there is an approximation algorithm for Vertex Cover in degree-{d + 2) graphs with performance 
ratio better than 1 + £2/2, then P = NP. 

4.3 Steiner Tree 

In the Steiner tree problem we are given a graph G = iy^E), with weights on the edges, and a 
subset C C y of vertices. We want to find the tree of minimum cost that contains all the vertices 
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of C. This problem is different from the minimum spanning tree problem because the tree is not 
required to contain the vertices inV — C, although it may contain some of them if this is convenient. 
An interesting special cases (called Metric Steiner Tree) arises when E is the complete graph and 
the weights are a metric. (That is, they satisfy the triangle inequality.) Without this restriction, 
it is not hard to show that the problem cannot be approximated within any constant. 

We describe a reduction from the Vertex Cover problem in bounded degree graphs to the Steiner 
Tree problem. The reduction is due to Bern and Plassmann |BP89j . 

We start from a graph G = {V, E), and we assume that G is connected.® We define a new graph 
G' that has \V\ + \E\ vertices, that is, a vertex [v] for each vertex w of G and a vertex [u, v] for each 
edge {u, v) of G. 

The distances in G' are defined as follows: 

• For every edge (n, v) £ E, the vertices [u] and [n, v] are at distance one, and so are the vertices 
[v] and [u,v]. 

• Any two vertices [u], [v] are at distance one. 

• All other pairs of vertices are at distance two. 

We let C be the set of vertices {[u, : {u,v) £ E}. This completes the description of the 
instance of the Steiner Tree problem. Notice that, since all the distances are either one or two, they 
satisfy the triangle inequality, and so the reduction always produces an instance of Metric Steiner 
Tree. 

Claim 10 // there is a vertex cover in G with k vertices, then there is a Steiner tree in G' of cost 
m + k — 1. 

Proof: Let S be the vertex cover. Consider the vertices {[v] : v € S} and the vertices {[u, : 
{u,v) E E}, and consider the weight-one edges between them. We have described a connected 
sub-graph of G' , because every vertex in {[u] : u £ S} is connected to every other vertex in the 
same set, and every vertex [n, v] is connected to a vertex in {[n] : n G 5}. Let us take any spanning 
tree of this subgraph. It has m + k — 1 edges of weight one, and so it is of cost m + k — 1, and it is 
a feasible solution to the Steiner Tree problem. □ 

Claim 11 // there is a Steiner tree in G' of cost < m + k, then there is a vertex cover in G with 
k vertices. 

Proof: Let T be a feasible Steiner tree. We first modify the tree so that it has no edge of cost 2. 
We repeatedly apply the following steps. 

• If there is an edge of cost 2 between a vertex [w] and a vertex [u,v], we remove it and add 
the two edges {[w], [u]) and ([n], [u,v]) of cost 1. 

• If there is an edge of cost 2 between a vertex [u, v] and a vertex [f , it'], we remove it and add 
the two edges ([tt, f], [v]) and {[v], [v,w]) of cost 1. 

®We proved inapproximability of Vertex Cover without guaranteeing a connected graph. Clearly, if we have an 
approximation algorithm that works only in connected graphs, we can make it work on general graphs with same 
factor. It follows that any inapproximability for general graphs implies inapproximability of connected graphs with 
the same factor. 
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• Finally, and this case is more interesting, if there is an edge of cost 2 between the vertices 
[u, v] and [w, z], we remove the edge, and then we look at the two connected components into 
which T has been broken. Some vertices [u, v] are in one component, and some vertices are 
in the other. This corresponds to a partition of the edges of G into two subsets. Since G is 
connected, we see that there must be two edges on different sides of the partition that share 
an endpoint. Let these edges be {u,v) and {vjw) then we can reconnect T by adding the 
edges {[u,v], [v]) and {[v], [v,w]). 

We repeat the above steps until no edges of cost two remain. This process will not increase the 
cost, and will return a connected graph. We can obtain a tree by removing edges, and improving 
the cost, if necessary. 

The final tree has only edges of weight one, and it has a cost < m + k — 1, so it follows that it 
spans <m + k vertices. The m vertices {[u, f] : {u,v) G E} must be in the tree, so the tree has < k 
vertices [v]. Let S be the set of such vertices. We claim that this is a vertex cover for G. Indeed, 
for every edge (n, v) in G, the vertex [u, v] is connected to vq in the tree using only edges of weight 
one, which means that either [u] or [v] is in the tree, and that either or v is in S. □ 

If we combine the reduction with the results of Theorem 121 we prove the following theorem. 

Theorem 12 There is a constant £3 such that if there is a polynomial time (1 + e^)- approximate 
algorithm for Metric Steiner Tree then P = NP. 

4.4 More About Independent Set 

In this section we describe a direct reduction from PCP to the Independent Set problem. This 
reduction is due to Feige et al. [FGL+Qd] . 

Let L be NP-complete, and ^ be a verifier showing that L £ FCPc,s[Q{n),r{n)]. For an 
input X, let us consider all possible computations of V^{x) over all possible proofs w; a complete 
description of a computation of V is given by a specification of the randomness used by V, the 
list of queries made by V into the proof, and the list of answers. Indeed, for a fixed input x, each 
query is determined by x, the randomness, and the previous answers, so that it is enough to specify 
the randomness and the answers in order to completely specify a computation. We call such a 
description a configuration. Note that the total number of configuration is at most 2'"(") • 2'i^^\ 
where n is the length of x. 

Consider now the graph Gx that has a vertex for each accepting configuration of V on input 
X, and has an edge between two configurations c, c' if c and c' are inconsistent, that is, if c and 
c' specify a query to the same location and two different answers to that query. We make the 
following claims. 

Claim 13 If x G L, then Gx has an independent set of size > c ■ 2^^"'\ 

Proof: If x S L, then there is a proof w such that V^{x) accepts with probability at least c, that 
is, there is a proof w such that there are at least c • 2^"^"^ random inputs that make V'^{x) accept. 
This implies that there are at least c-2^("') mutually consistent configurations in the graph Gx, and 
they form an independent set. □ 

Claim 14 If x ^ L, then every independent set of Gx has size < s ■ 2^'^'^\ 
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Proof: We prove the contrapositive: we assume that there is an independent set in Gx of size 
> s-2''("),and we show that this imphes x £ L. Define a witness w as follows: for every configuration 
in the independent set, fix the bits in w queried in the configuration according to the answers in the 
configurations. Set the bits of w not queried in any configuration in the independent set arbitrarily, 
for example set them all to zero. The s ■ 2^^"^ configurations in the independent set correspond to 
as many different random strings. When V^{x) picks any such random string, it accepts, and so 
V^{x) accepts with probability at least s, implying 2; G L. □ 

It follows that if there is a p-approximate algorithm for the independent set problem, then every 
problem in PCPc,s[^(™)) ^C'^)] can be solved in time poly(n, 2''("^^''(")), provided c/s < p. 

From the PCP Theorem we immediately get that there cannot be a p-approximate algorithm 
for the independent set problem with p < 2 unless P = NP, but we can do better. 

Suppose that y is a (0(log n), 0(l))-restricted verifier for an NP-complete problem, and that 
V has soundness 1/2 and completeness 1. Define a new verifier V' that performs two independent 
repetitions of the computation of V, and that accepts if and only if both repetitions accept. Then 
V' has clearly soundness 1/4 and completeness 1, and it is still (O(logn), 0(l))-restricted, thus 
showing that even an approximation better than 4 is infeasible. If we repeat V a constant number 
of times, rather than twice, we can rule out any constant factor approximation for the independent 
set problem. 

In general, a verifier that makes k{n) repetitions shows that L S FCFii^2''(")[0{k{n) ■ 
log n), 0(A;(n))], and the reduction to Independent Set produces graphs that have 2^^^^"'^'^°^"'^ 
vertices and for which 2'^^"')-approximate algorithms are infeasible. If we let k{n) = logn, then the 
graph has size N = 2'^(('°s") ^ and the infeasible ratio is n, which is 2^^^^°^^\ So, if we have an 
algorithm that on graphs with N vertices runs in polynomial time and has an approximation ratio 
2o(ViogAf)^ then we have an 0(re'^(i°g")) algorithm to solve 3SAT, and NP C QP. More generally, 
by setting k{n) = (logn)*^(^\ we can show that if there is an e > such that Independent Set can 
be approximated within a factor 2'^^*^^°^"'^ ^-^ then NP C QP. 

Finally, using random walks in expander graphs as in |IZ89j . it is possible to use the PCP 
theorem to show that, for every k{n), NP = PCP^^ ^^2k(")[0{k{n) + logn),0{k{n))]. If we choose 

k{n) = logn, then, in the reduction, we have a graph of size 2'^('^("')+'°S"') = n'^^^^ for which an 
approximation ratio of n is infeasible. This shows the following result. 

Theorem 15 There is a constant c > 1 such that if there is a polynomial time n'^ -approximate 
algorithm for Independent Set then P = NP. 

5 Optimized Reductions and PCP Constructions 

In this section we give an overview of tighter inapproximability results proved with a combination 
of optimized PCP constructions and optimized reductions. 

5.1 PCPs Optimized for Max SAT and Max CUT 

In the reduction from a (0(log n), q)-restricted verifier to Max SAT we lost a factor that was 
exponential in q. A more careful analysis shows that the loss depends on the size of the CNF 
that expresses a computation of the verifier. To prove stronger inapproximability results for Max 
SAT one needs a verifier that, at the same time, has a good separation between soundness and 
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completeness and a simple acceptance predicate. Progress along these lines is reported in |BGLR93l 
IFKMIIR^IRH?^ . 

The optimal inapproximability result for Max 3SAT was proved by Hastad [HasOlj . based on 
the following PCP construction. 

Theorem 16 (Hastad [HasOl]) For every e > 0, NP = PCPi_£ i/2+e[0(logn),3]. Furthermore, 
the verifier behaves as follows: it uses its randomness to pick three entries i,j, k in the witness and 
a bit b, and it accepts if and only if Wi® Wj ® Wk = b. 

The furthermore part immediately implies that for every problem in NP, say, SAT for concrete- 
ness, and for every e > there is a reduction that given a 3CNF formula ip constructs a system of 
linear equations over GF{2) with three variables per equation. If ip is satisfiable then there is an 
assignment to the variables that satisfies a 1 — e fraction of the equations, and if (p is not satisfiable 
then there is no assignment that satisfies more than a 1/2 + e fraction of equations. In particular, 
it is not possible to approximate the Max E3LIN-2 problem to within a factor better than 2 unless 
P = NP. 

To reduce an instance / of Max E3LIN-2 to an instance (pi of Max 3SAT, we simply take 
every equation Xi © Xj © Xfc = 6 in / and express it as the conjunction of 4 clauses, then ipj is the 
conjunction of all these clauses. Let m be the number of equations in /, then ipj has 4m clauses. If 
> m(l — e) of the E3LIN equations could be satisfied, then > 4m(l — e) of the clauses of ipi can be 
satsfied, using the same assignment. Conversely, if it is impossible to satisfy more than m(l/2 + e) 
equations in the E3LIN instance / then it is impossible to satisfy more than 3.5m + em clauses in 
ipj. An approximation algorithm for Max 3SAT with a performance ratio better than 8/7 allows 
to distinguish between the two cases, if s is small enough, and so such an algorithm is impossible 
unless P = NP. 

Theorem 17 If there is an r-approximate algorithm for Max 3SAT, where r < 8/7, then P = NP. 

A similar "local" reduction is possible from Max E3LIN-2 to Max CUT. For the purpose of 
the reduction it is easier to think of Max CUT as the boolean constraint satisfaction problem 
where we have Boolean variables xi, . . . ,Xn (one for every vertex in the graph) and constraints 
of the form Xi ^ xj (one for every edge {i,j) in the graph); we call constaints of this form "cut 
constraints." To reduce a Max E3LIN-2 instance / to a Max CUT instance Gj, we take each 
constraint Xj © Xj © x^ = 5 of / and we construct a collection of cut constraints over Xi,Xj,Xk, 
plus auxiliary variables. The construction (from [TSSWOQJ) is such that an assignment to Xi,Xj,Xk 
that satisfies the E3LIN-2 constraint with 6 = can be extended to the auxiliary variable so that 
it satisfies 9 of the cut constraints, while an assignment that contradicts the E3LIN-2 constraint, 
no matter how it is extended to the auxiliary variables, satisfies at most 8 cut constraints (and 8 
constraints can always be satisfied by setting the auxiliary variables appropriately). When 6 = 1, a 
similar construction exists and the number of satisfied constraints is 8 versus 7 (instead of 9 versus 
8). In E3LIN-2 instances that are derived from Hastad's PCP there are as many equations with 
right-hand side as equations with right-hand side 1. Given all these facts, simple computations 
show that 

Theorem 18 If there is an r-approximate algorithm for Max CUT, wherer < 17/16, thenP = NP. 
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The best known approximation algorithm for Max CUT is due to Goemans and Wihiamson 
ICirWQBj . and its performance ratio is 1/a where a ~ .878. Khot et al. [KKMOnlj describe a 
PCP system that may prove that the algorithm of Goemans and Williamson is best possible. In 
particular, if the "unique game conjecture" is true'' and if a conjecture about extremal properties 
of monotone functions is true, then there is no polynomial time approximation algorithm for Max 
CUT with performance ratio better than l/a, unless P = NP. 



5.2 PCPs Optimized for Independent Set 

The graph in the reduction of Feige et al. |FGL"'"96 has a vertex for every accepting computation. 



This implies that the important parameter to optimize (in order to derive strong inapproximability 
results for Independent Set) is not the number of queries but rather the number of accepting 
computations. This intuition motivates the following definition. 

Definition 19 (Free Bit Complexity) We say that a {r{n), q{n)) -restricted verifier V for a lan- 
guage L has free bit complexity at most f{n) if, for every input x and every choice of the random 
input, there are at most 2^(") possible answers to the q{n) queries made by the verifier that would 
make it accept. 

In the reduction from PCP to Independent Set, if we start from a (r(n), g(n))-restricted verifier 
of free bit complexity /(n), then we construct a graph with < 2^*^"')"'"-^(") vertices, so that, all other 
things being equal, a lower free bit complexity yields a smaller graph and so a stronger relation 



between inapproximability and graph size. A further inspection of the reduction of |FGL"'"96 
shows that what really matters is the ratio /(n)/log2(l/s(n)), where s{n) is the soundness of the 
verifier, assuming that the verifier has completeness 1. This measure is called the amortized free 
bits complexity of the verifier. 

Theorem 20 (Optimized FGLSS Reduction [BGS98]) Suppose that an NP -complete prob- 
lem has an {O (log n),0 {log n)) -restricted verifier of amortized free bit complexity f, and suppose 
that, for some e > 0, there is an n'^'^^^^^^^^ -approximate algorithm for Independent Set. Then 
NP = ZPP. 



In order to prove strong inapproximability results it is then enough to provide a version of the 
PCP Theorem with a verifier having low amortized free bit complexity. Hastad proved the stronger 
possible result, stating that an arbitrarily small amortized free bit complexity is sufficient. 

Theorem 21 (Hastad [Has99j) For every 6 > 0, 3SAT has an {O {log n),0{l)) -restricted veri- 
fier of amortized free bit complexity < 6. Therefore, assuming ZPP ^ NP, for every e > there is 
no n^~^ -approximate algorithm for the Independent Set problem. 

The proof of this result was substantially simphfied in |STnni IhWHT] . 

If a graph G = {V, E) has maximum degree d, then a maximal independent set contains at least 
|y|/(d+l) vertices, and so is a (d+ l)-approximate solution. This can be slightly improved, and an 
0(dloglogd/logd)-approximate algorithm is also known (KMS98. AK98I IVis96l lHal98j . Trevisan 
ITreOlj proves that no ((i/2*^*^^^°S'^^)-approximate algorithm exists unless P = NP. 

^The unique game conjecture, posed by Khot |Kho02b] . is about the existence of a PCP system for NP with 
certain properties. See also Section |H| 
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6 An Overview of Known Inapproximability Results 



After the overview of techniques given in the previous sections, we devote this section to an overview 
of known inapproximabihty results for a few selected problems. Some more inapproximability 
results for specific problems are also discussed in Section [7.41 below. 



6.1 Lattice Problems 

Let B = {vi, . . . , v„} be a set of linearly independent vectors in M". The lattice generated by B 
is the set of all points of the form ajV, where are integers. For example, suppose n = 2 
and B = {(1/2, 0), (0, 1/2)}. Then the lattice generated by B contains all points in whose 
coordinates are half-integral. 



Shortest Vector Problem 

The Shortest Vector Problem, abbreviated SVP, is the problem of finding the shortest non-zero 
vector of a lattice, given a base for the lattice. By "shortest vector," we mean the vector of smallest 



(.2 norm, where the ^2 norm of a vector v = (vi, . . . , Vn) is | |v| I2 = \jYlii • 

SVP is a non-trivial problem because the shortest vector may not be any of the basis vectors, 
and may in fact be a complicated linear combination of the basis vectors. For example, suppose that 
the basis is {(3, 4), (4, 5)}. Then one can immediately see that a shorter vector (1, 1) can be derived 
by subtracting one base vector from the other; it takes some more attention to see that a shortest 
vector can be derived as (0, 1) = 4-(3, 4) — 3-(4, 5). Of course the complexity of the problem escalates 
with the number of dimension and, perhaps surprisingly, the best known approximation algorithms 
for SVP achieve only an exponential approximation as a function of n, and even achieving such 
an approximation is highly non-trivial |LLL82| ISch87| lAKSOlj . The best known approximation 
algorithm has performance ratio 2'-^("'(^°siog"')/iog"') |AKSnij . On the other hand, Aharonov and 
Regev AR04j prove than an 0(-yn)-approximation can be computed in NP n coNP,^ and so the 
problem of finding 0(y^)-approximate solutions cannot be NP-hard unless NP = coNP. 

Interest in the SVP is motivated in part by a result of Ajtai jAjtOGj, showing that if ap- 
proximation of SVP cannot be done in polynomial time, for a certain constant c, then there is a 
problem in NP that does not have any algorithm running in polynomial time on average. Ajtai's 
result was the first one to show that a problem in NP is hard on average provided that another 
problem in NP is hard on worst-case. Even more interestingly, Ajtai and Dwork |AD97j presented 
a public key cryptosystem that is secure provided that a version of SVP is hard to approximate. 
The results of Ajtai and Dwork |Ajt961 IAD97j have been later improved by Regev |Reg03| and by 
Micciancio and Regev |MB,n4j . Currently, the cryptosystems and average-case complexity results 
of .A.iMl lA 1)971 |Reg03| IMH,n4j depend on the worst -case complexity of problems known to be in 



NPncoNP, but it is conceivable that the analyses could be improved so that they rely on problems 
that are NP-hard to approximate. If so, there would be average-case hard problems, one-way func- 
tions and even private- key encryption be based on the assumption that P 7^ NP (or, more precisely, 
that NP 2 BPP). Towards this goal, one needs, of course, strong inapproximability results for 
SVP. 



**We have not quite defined what it means to have an approximation algorithm in NP PI coNP. See |AR,n4| for a 
discussion. 
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The first inapproximability result for SVP was proved by Micciancio [MicOlj . who showed that 
there is no (\/2 — e)-approximate algorithm for SVP for any e > 0, unless RP = NP. This 
result has been substantially improved by Khot |Kho04aj . who proves that no 0(l)-approximate 
algorithm for SVP exists unless RP = NP, and that no 2^^°^"'^'^ ^ ^-approximate algorithm exists 
unless NP C QRP, for every e > 0. 

Closest Vector Problem 

In the Closest Vector Problem, abbreviated CVP, we are given a lattice and a vector, and the 
problem is to find the lattice element that is closest (in ^2 norm) to the given vector. The approx- 
imation algorithms that are known for the SVP can be adapted to approximate CVP within the 
same ratios |LLL821 ISch87l lAKSOlj , and the techniques of Aharonov and Regev also show that 
achieving an 0(-^/n) approximation is in NP n coNP |AR,fl4j . 

The best inapproximability result for CVP is due to Dinur et al., who show that there can be 
no n^/°(^°siogn).appi.oximate algorithm unless P = NP (DKRSnH; . 

Other Norms 

Interesting variants of the SVP and CVP arise when we measure the length of vectors using other ip 
norms, where the ip norm of a vector v = (vi, . . . , Vn) is | |v| |p = - \vi\P)^^^; the problems are then 
denoted SVPp and CVPp, respectively. Of interest is also the case of the 

ioo norm 1 1^| |oo — maxj \ vi\, 

that defines the problems SVPqo and CVPqo- 

Any £p norm is a 0{^/n) approximation of the £2 norm, so the exponential approximation of 
|LLL82I ISch871 lAKSOIj also apply to other £p norm. 

For SVPp, Micciancio's result |Mic01j proves a hardness of approximation within 2^/^ — e for 
any £p norm and £ > 0. For every 6 > and for every sufficiently large p, Khot |Khon3j proves 
hardness p^^^. The new result of Khot |Kho04aj rules out r-approximate algorithms for SVPp for 
every constant r and every 1 < p < 00, assuming RP ^ NP, and rules out 2^^°^"'''^'^^ '^-approximate 
algorithm for every 1 < p < 00 and £ > 0, assuming NP ^ QRP. Micciancio's result remains the 
strongest known one for the £1 norm. 

For CVPp, the hardness result of Dinur and others jDKE,Sfl3] . ruling out n^/'^(^"s^°sn)_ 
approximate algorithms, assuming P 7^ NP applies to any £p norm 1 < p < 00. 

Regarding the £00 norm, Dinur |l)inn2j proves that there is no n''^^^^"^ -approximate algo- 
rithm for SVPoo and CVP 00; assuming P ^ NP. 

6.2 Decoding Linear Error-Correcting Codes 

Let F be a field and A he a n x k matrix over F. The mapping C{x) = A ■ x is a linear error- 
correcting code with minimum distance d if for every two disting inputs x ^ x' €z ¥^ the vectors 
C(x) and C(x') differ in at least d entries. It is easy to see that this happens if and only if for every 
X the vector C(x) has at least d non-zero entries. In turn, this happens if and only if every d 
rows of A are linearly independent. 

Error-correcting codes are used for the transmission (and storage) of information using unreli- 
able media. Suppose that a sender has a message x and that he wants to send it to a recipient 
using an unreliable channel that introduces transmission errors, and suppose also that the sender 
and the recipient know a linear error-correcting code C : F'^ — > F" of minimum distance d. Then 
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the sender can compute C{x) and send it to the recipient through the channel. Due to transmission 
errors, the recipient receives a string y € F", that may be different from C{x). Assuming that fewer 
that d/2 errors occurred, there is a unique string in the range of C() that is closest to y, and this 
string is C{x). If there is confidence that no more than d/2 errors occurred, then the recipient can 
reconstruct x this way. 

Clearly, the larger is d the higher is the reliability of the system, because more transmission 
errors can be tolerated. Also, for fixed d and k, the smaller is n the more efficient is the transmission. 
Therefore, we would like to construct codes, that is, matrices, with large d and small n for given 
k. Asymptotically, we would like to find large p, 6 such that for every n k = pn and d = 5n. It 
is beyond the scope of this paper to give an overview of known results about this problem. The 
reader is referred to a coding theory book like |vL99j for more information. 

The natural optimization problems associated with linear codes are the following: Suppose we 
are given a linear code, specified by the encoding matrix A, can we compute its minimum distance? 
And given a good code C of large minimum distance and a string y, can we find the decoding x 
such that C(x) is closest to y? 

The first problem is similar to SVP, and it is formally defined as follows: given a matrix 
A e F"^^ where F is a field, find the non-zero vector x S F'^ such that A ■ x has the smallest 
number of non-zero entries. Dumer, Micciancio and Sudan |DMS03j prove that, assuming that 
NP 2 QRP, the problem cannot be approximated within a factor 2^'°^"") ^. Their result holds 
even if the fielf F is restricted to be {0, 1}. The second problem. Nearest Codeword, is similar to 
CVP. Formally, our input is a matrix A G F"^'^ and a vector y G F", and the goal is to find a vector 
x G F*"' such that the number of entries where A ■ x and y differ is minimized. Arora et al. |ABSS97] 
prove that the problem cannot be approximated within a factor 2('°s") , assuming NP 2 QP- 

6.3 The Traveling Salesman Problem 

In the Traveling Salesman Problem (TSP) we are given a complete undirected graph G = {V, E) 
with non-negative weights w : ^ M on the edges. We think of the vertices as being "cities" and 
the weight of the edge (u, v) as the "distance" between two cities. The goal is to find a route that 
touches each city exactly once and that has minimal total length. 

If the distances are allowed to be arbitrary, then it is easy to see that even deciding whether the 
optimum is zero or not is an NP-hard problem, and so no approximation is possible |SG76j . One, 
however, is typically interested in instances where the distances satisfy the triangle inequality, and 
thus describe a metric on the set of cities. This special case is called the Metric TSP, or ATSP. 
The metric TSP is equivalent to the variation of TSP with arbitrary distances in which the route 
is required to pass through each city at least once, but it is allowed pass more than once through 
some city. 

Almost 30 years ago, Christofides devised a 3/2-approximate algorithm for Metric TSP |Chr76j . 
and the performance ratio of his algorithm has never been improved upon. 

Papadimitriou and Yannakakis |PY93j present an approximation-preserving reduction from Max 
3SAT-(i to Metric TSP. The reduction, together with other reductions seen in the previous section 
and with the PCP Theorem, implies that there is no PTAS for Metric TSP unless P = NP. More 
specifically, there is a constant stsp > such that if there is a polynomial time (1 -|- etsp)- 
approximate algorithm for Metric TSP then P = NP. The value of etsp implied by the PCP 
Theorem and the reductions is extremely small, and there has been work devoted to derive stronger 
inapproximability results. Papadimitriou and Vempala |PVOO) . using Theorem (Hastad's three- 
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query PCP construction |Has01j l and a new reduction show that there is no polynomial time 
algorithm for Metric TSP with performance ratio better that 220/219, unless P = NP. 

Grigni, Koutsopias and Papadimitriou show that Metric TSP has a PTAS in the special case 
in which the metric is the shortest path metric of a planar graph KjKPQBj . Arora [Aro98b and 
Mitchell |Mit99j show that there is a PTAS in the special case in which cities are points in and 
distances are computed according to the £2 norm. Arora's PTAS also works for other metrics 
and in higher dimension, with running time doubly exponential in the number of dimensions. 
Trevisan llVeOOl shows that, if P 7^ NP, then Metric TSP in R'^^") with an metric has no PTAS 
if d{n) = i7(logn). This implies that the double exponential dependency of the running time on 
the number of dimensions in Arora's PTAS cannot be improved unless every problem in NP has 
sub exponential algorithms. 

Another interesting version is Asymmetric TSP, in which the "distance" from a city u to a 
city V may be different from the distance from v to u. The Asymmetric ATSP is the restriction 
of Asymmetric TSP to asymmetric "distance" functions that satisfy the triangle inequality. A 
polynomial 0(log n)-approximate algorithm for Asymmetric ATSP is known |F(TM82j . Papadim- 
itriou and Vempala PV0()1 prove hat there is no polynomial time algorithm for Metric TSP with 
performance ratio better that 117/116,^ unless P = NP. It remains an interesting open question 
to rule out constant factor approximation for Asymmetric ATSP, or to devise a constant-factor 
approximation algorithm. 



6.4 Coloring Problems 

In the graph coloring problem we are given an undirected graph G = {V^ E) and we want to label 
each vertex with a "color" so that every edge has endpoints of different colors. The goal is to use 
as few colors as possible. 

Feige and Kilian prove that, if ZPP 7^ NP, then, for every e, there is no n^~^-approximate 
algorithm for this problem, where n is the number of vertices. 

An interesting special case of the problem is to devise algorithms that color a 3-colorable graph 
with a minimum number of colors. (On other graphs, the algorithm may have an arbitrary behav- 
ior.) Blum and Karger ||BK97) (improving previous work by Karger, Motwani and Sudan jKMS98j 
and by Wigderson Wig83| ), prove that there is a polynomial time algorithm that colors every 3- 



colorable graph with at most n^/^^-polylog(n) colors. Khanna, Linial and Safra |KLS93j prove that 
there is no polynomial time algorithm that colors every 3-colorable graph using at most four colors, 
unless P = NP, and this is still the strongest known inapproximability result for this problem. This 
is one of the largest gaps between known approximation algorithms and known inapproximability 
results for a natural and well-studied problem. 

It appears that methods based on probabilistically checkable proofs could be applied to the 
problem of coloring 3-colorable graphs, provided that one uses a different defnition of "soundness." 
Guruswami, Hastad and Sudan |GHS02j describe a type of PCP that could be used to prove 
hardness of 3-coloring. They are able to construct such PCPs with parameters that are sufficient 
to prove intractability for the Hypergraph Coloring problem. In the Hypergraph Coloring problem 
we are given a hypergraph H = (y,E), where E is a, collection of subsets of V, and we want to 

®The conference version of |PVOO| claimed that no polynomial (129/128 — e)-approximate algorithm for Metric 
TSP and no (42/41 — e)-approximate algorithm for Asymmetric ATSP exists for any e > 0, assuming P 7^ NP. The 
stronger claim was due to an error in the proof. The corrected version of the paper, with the weaker inapproximability 
results, is available from the authors. 



21 



color the vertices with a minimum number of colors so that there is no edge that contains vertices 
all of the same color. A hypergraph is r-uniform if every hyperedge contains exactly r vertices. 
An undirected graph is a 2-uniform hypergraph. Guruswami et al. |GHS02j show that, if P 7^ NP, 
there is no polynomial time algorithm that, given a 2-colorable 4-uniform hypergraph, finds a 
feasible coloring that uses a constant number of colors. Dinur, Regev and Smyth |DRS02j (see also 
|Khon2aj ) prove the same result for 3-uniform hypergraphs. 

Regarding /c-colorable graphs for large constant k, Khot |Kho01j proves that there is no poly- 
nomial time algorithm that, given a /c-colorable graph, colors it with colors. 



6.5 Covering Problems 

Using Hastad's POP construction (Theorem ll6() and the reduction of [FGL"'"96 , one can prove that 



there is no polynomial time algorithm for the Vertex Cover problem with performance ratio better 
than 7/6. 

This result has been improved by Dinur and Safra |DS02j . who prove that there is no approxi- 
mation algorithm for Vertex Cover with performance ratio smaller than W^/b — 21 = 1.36 . . .- unless 
P = NP. A simple 2-approximate algorithm has been known to exist for thirty years |Gav74j . If 
the unique games conjecture is true, then 2 is the best possible performance ratio for polynomial 
time algorithms, unless P = NP KR03]. (See also Section |H1 below.) 

In the Set Cover problem we are given a collection of subsets 5i , . . . , Sm of a set U such that 
Uj Si = U. The goal is to find a smallest set / C {1, . . . , m} such that Uie/ ~ ^ ■ problem 
can be approximated within a factor ln|C/| -|- 1, using either a greedy or a primal-dual algorithm 
|Joh741 fLov75[ I^hv79j . More precisely, if every set is of size at most k, then the approximation 
ratio of the algorithm is at most H^, defined as Hj^ = l-|-l/2-|-...-|-l/A; < l-|-lnA:. Feige |Fei98) 
shows that there cannot be a (1 — e) In |f7| approximate algorithm, for any e > 0, unless NP C QP. 
Trevisan [TreOlj notes that Feige's proof also implies that there is a constant c such that the 
Set Cover problem with sets of size k (where k is constant) has no (In A; — clnln /c)-approximate 
algorithm unless P = NP. 

In the Hitting Set problem we are given collection of subsets 5i, . . . , Sm of a set U . The goal 
is to find a smallest set 5 C [/ such that S'j n 5 / for every i = 1, . . . ,m. It is easy to see 
that the Set Cover problem and the Hitting Set problem are equivalent, and that there is an r{k)- 
approximate algorithm for the Set Cover problem, where k is the size of the largest set, if and only 
if there is an r(t)-approximate algorithm for the Hitting Set problem, where t is an upper bound to 
the number of sets to which an element of U may belong. Similarly, there is an r(t)-approximate 
algorithm for the Set Cover problem, where t is an upper bound to the number of sets to which an 
element of U may belong if and only if there is an r(A;)-approximate algorithm for the Hitting Set 
problem, where where k is the size of the largest set. In particular, it follows from the results of 
|.Toh74| rLov75| IChv79| IFei98j that there is a (1 -|- In m)-approximate algorithm for Hitting Set, and 
that there is no (1 — e) • In m-approximate algorithm with e > unless NP C QP. 

Consider now the Hitting Set problem restricted to instances where every set is of size at most 
k. This is equivalent to the Set Cover problem restricted to instances where every element of U 
may appear in at most k sets. Another equivalent formulation of the problem is as the Vertex 
Cover problem in fc-uniform hypergraphs. (In particular, it is just the vertex cover problem in 
graphs when k = 2.) A simple generalization of the 2-approximate algorithm for Vertex Cover 
yields a fc-approximate algorithm for this problem |Hoc82j . Dinur et al. |DGKR03] prove there is 
no (A; — 1 — e)-approximate algorithm, e > 0, unless P = NP. Assuming that the unique games 
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conjecture holds, even {k — e)-approximate algorithms are impossible |KR03j . 



7 Other Topics 

7.1 Complexity Classes of Optimization Problems 

Prior to the discovery of the connection between probabilistically checkable proofs and inapproxima- 
bility, research on inapproximability focused on the study of approximation-preserving reductions 
among problems, and on classes of optimization problems and completeness results. 

The first definitions of approximation-preserving reductions appear in work by Paz and Moran 
|PM8H and by Ausiello, D'Atri and Protasi |ADP8n| . 

Orponen and Manilla |()M87j define the class NPO, and show that certain (artificial) problems 
are NPO-complete under approximation-preserving reductions. Kann |Kan93j finds problems that 
are complete for NPO-PB, the restriction of NPO to problems whose objective function is polyno- 
mially bounded in the length of the input. Crescenzi and Panconesi |CP91j study the class APX 
of problems that admit an 0(l)-approximate algorithm. They show the existence of (artificial) 
complete problems in APX with respect to reductions that preserve the existence of approximation 
scheme. It is easy to see that, if P 7^ NP, there are problems in APX that do not have approxima- 
tion schemes, and so, in particular, no APX-complete problem can have a PTAS unless P = NP. 
These results were meant as proofs-of-concept for the program of proving inapproximability results 
by proving completeness results under approximation-preserving reductions. None of these papers, 
however, proves inapproximability results for natural problems. 

A very different, and more productive, direction was taken by Papadimitriou and Yannakakis 
|PY91j . They follow the intuition that Max 3SAT should be as central a problem to the study of 
constant factor approximation as 3SAT is to the study of decision problems in NP. Papadimitriou 
and Yannakakis identify a class of optimization problems, called Max SNP, that can be defined using 
a certain fragment of logic. They show that Max SNP includes Max 3SAT and other constraint sat- 
isfaction, and they prove that Max 3SAT is complete for Max SNP under approximation-preserving 
reductions. It follows that Max 3SAT has a PTAS if and only if every problem in Max SNP has 
a PTAS. More importantly, Papadimitriou and Yannakakis present several reductions from Max 
3SAT using approximation-preserving reductions, including ones that we presented in earlier sec- 
tions. The reductions proved in jPY91j . and in other papers such as |MP89[ IPY93j . implied that 
if one could rule out approximation schemes for Max 3SAT (or, equivalently, for any other prob- 
lem in Max SNP), then inapproximability results would follow for many other problems. As we 
discussed earlier, the PCP Theorem finally proved the non-existence of approximation schemes for 
Max 3SAT (assuming P 7^ NP), and the reductions of |PY91[ IMP89L IP ¥93; played the role that 
they were designed for. Around the same time, in another interesting result, Berman and Schnitger 
|BS92j proved that if Max 2SAT does not have a probabilistic PTAS (or equivalently, in light of 
|PY91j . if Max 3SAT does not have a probabilistic PTAS), then there is a c > such that there 
is no n'^-approximate algorithm for Independent Set. The reduction of Berman and Schnither and 
the PCP Theorem imply that there is a c > such that there is no n^-approximate algorithm 
for Indepedent Set, assuming RP / NP. As we discussed in Section Arora et al. ALM+98] 



achieve the same conclusion from the assumption that P 7^ NP, by improving the soundness of the 
PCP construction using random walks on an expander. As discussed in |AFWZ95 , one can see the 



reduction of Arora et al. ALM^98 as a "derandomized" version of the reduction of BS92I 
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The most interesting results in the paper of Papadimitriou and Yannakakis |PY91j were the 
approximation preserving reductions from Max 3SAT to other problems, but it was the idea of 
defining classes of optimization problems using fragments of logical theories that caught other 
people's immagination. Other papers followed with logic-based definitions of optimization classes 
having natural complete problems, such as the work of Kolaitis and Thakur |KT94l IKT95j and of 
Panconesi and Ranjan |PR,9f)j . 

The connection between proof checking and approximation and the POP theorem allowed re- 
searchers to prove inapproximability results directly, without using notions of completeness, and 
gave central importance to reductions between specific problems. 

Khanna, Motwani, Sudan and Vazirani |KMSV9'9] revisited the issue of completeness in ap- 
proximation classes, and gave a definitive treatment. Khanna et al. |KMSV9'9] show how to use 
PCP to prove natural problems complete in their respective classes. For example they show that 
Max 3SAT is complete in APX, and that Independent Set is complete in poly-APX, the class of 
problems that are n'^-approximable for some c > 0, where n is the length of the input. Crescenzi et 
al. |('KST99| address some finer points about what definition of reduction should be used to prove 
general completeness results. 

An interesting related line of work, developed by Chang and others |(XirL97[ ICha96| l(]KST99] , 
is a characterization of the complexity of approximation problems in terms of "query complexity" 
to NP oracle. To see the point of this work, consider for example the problem of approximating 
Max 3SAT within a 1.1 versus a 1.01 factor. Both problems are NP-hard, and both problems can be 
solved in polynomial time if P = NP, so in particular if we had a polynomial time 1.1 approximate 
algorithm for Max 3SAT we could also design a polynomial time 1.01-approximate algorithm that 
uses the 1.1-approximate one as a subroutine. If we follow the proof of this result, we see that the 
hyptothetical 1.1-approximate algorithm is used more than once as a subroutine. Is this necessary? 
The answer turns out to be YES. Roughly speaking, computing a 1 + e approximation is equivalent 
to answering 0(l/e) non-adaptive queries to an NP oracle, and the power of what can be computed 
with k queries to an NP oracle is strictly less than what can be done with k + 1 queries (unless 
the polynomial hierarchy collapses). In contrast, in Independent Set, for every two constants 
1 < ri < r2, we can design an ri-approximate algorithm for Independent Set that uses only once 
a an r2-approximate algorithm. This property of the Independent Set problem (that different 
constant factor approximations have the same complexity) is called self-improvability. From the 
point of view of query complexity, computing a constant-factor approximation for Independent Set is 
equivalent to answering 0(logn) queries to an NP oracle. A consequence of these characterizations 
of approximation problems using query complexity is, for example, that one can prove that the Set 
Cover problem is not self-improvable unless the polynomial hierarchy collapses |(]KST99] . a result 
that it is not clear how to prove without using this machinery. 

7.2 Average-Case Complexity and Approximability 

Feige |Fein2bj proposes to use the conjectured average-case complexity of certain distributional 
problems^^ in order to prove inapproximability results. More specifically, Feige consider the 3SAT 
problem, together with the distribution where an instance with n variables and cn clauses, with 
c being a large constant, is picked uniformly at random from all instances with n variables and 
cn clauses. For sufficiently large constant c, such instances have an extremely high probability of 

^'^A distributional problem is a decision problem equipped with a probability distribution over the inputs. 
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being unsatisfiable, but is seems difficult to certify that specific instances are unsatisfiable. Feige 
considers the assumption^^ that, for every choice of c, there is no polynomial time algorithm that 
on input a satisfiable formula never outputs "UNSAT," and that on input a formula picked from 
the above distribution outputs "UNSAT" with high probability. (The probability being computed 
over the distribution of formulas.) 

Under this assumption, Feige shows that there is no (4/3 — e)-approximate algorithm for the 
Minimum Bisection problem, for any e > 0. The Minimum Bisection problem is defined as follows: 
we are given an undirected graph G = {V,E), with V having even cardinality, and our goal is to 
find a partition of V into two subsets S, S of equal size such that a minimum number of edges 
crosses the partition. An approximation algorithm due to Feige and Krautghamer |FKn2j . and it 
achieves a factor of 0((log n)'^). This has been recently improved to 0((log n)^'^) by Arora, Rao and 
Vazirani ARV04 . No inapproximability result was known for this problem prior to |Fei02bj . Khot 
|Khon4b] recently proved that the Minimum Bisection problem does not have an approximation 
scheme assuming that NP is not contained in sub-exponential time. More precisely, Khot shows 
that for every e > there is a constant > such that there is no (1 + C£)-approximate algorithm 
for Minimum Bisection unless 3SAT can be solved in time 2'-^^"'^\ 

Alekhnovich [AleDSj defines other distributional problems (related to E3LIN) and shows that 
certain problems in coding theory are hard to approximate provided that the distributional problem 
is hard on average. 

7.3 Witness Length in PCP Constructions 

The proof of the PCP theorem shows that for every NP-complete problem, for example 3SAT, 
we can encode a witness in such a way that it can be tested with a constant number of queries. 
How long is that encoding? The original proof of the PCP theorem shows that the size of the 
encoding is polynomial in the number of variables and clauses of the formula. Improvements 
in |PS94[ IFS95j shows that one can achieve cubic encoding length in the PCP Theorem with a 
moderate query complexity, or encoding length A^^"*""^, where N is the size of the formula, using 
query complexity that depends on e. Recent work shows that the witness needs only be of nearly 
linear |HS001 IBSSVW031 lBSGH+04| length in the size of the formula. It is conceivable that the 
witness could be of size linear in the number of clauses of the formula. 

The question of witness length in PCP construction is studied mostly for its connections to 
coding theory, but it has an interesting connection to approximability as well. 

If we let m denote the number of clauses and n the number of variables in a 3SAT instance, it 
is believed that the satisfiability problem cannot be solved in time 2°^"\ where n is the number of 
variables, even in instances where m = 0(n).^^ Suppose now that there were a proof of the PCP 
Theorem with a witness of linear size. Then this can be used to show that there is some e > such 
that a (1 + e)-approximate algorithm for Max 3SAT running in 2°^"^ time implies an algorithm 
that solves 3SAT in 2°("') time. Since such a consequence is considered unlikely, we would have to 
conclude that no sub-exponential algorithm can even achieve a good approximation for Max 3SAT. 
Conversely, a (1 -f o(l))-approximate algorithm for Max 3SAT running in 2"^") time would provide 
some evidence that the proof length in the PCP Theorem cannot be linear. 

^^He does not formulate it as a conjecture. 

^^In fact, Impagliazzo, Paturi and Zane I1PZ01| prove that, roughly speaking, if 3SAT has a 2°*"' time algorithm 
that works only on instances where m = 0{n), then there is also a 2°*^"' time algorithm for 3SAT that works on all 
instances. 
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7.4 Typical and Unusual Approximation Factors 

Looking at the inapproximability results proved in the 1990s using the PCP Theorem, it is easy 
to see a certain pattern. For several problems, such as Max 3SAT, Vertex Cover, Metric TSP, 
Metric Steiner Tree, Max CUT, we know that a polynomial time constant factor approximation 
exists, and there is an inapproximability result proving that for some e > there is no (1 + e)- 
approximate algorithm. For other problems, like Clique, Independent Set, and Chromatic Number, 
we can prove that there is an c > such no n'^-approximate algorithm exists, where n is the size 
of the input. Then there are problems like Set Cover that are approximable within an O(logn) 
factor but not within a o(log n) factor, and finally there are some problems for which we can show 
an inapproximability result of the form 2'^^°^"^'' for some constant c > 0, although we believe that 
the right lower bound is for some c > 0. 

For maximization problems, known results either prove that no PTAS exists (while a constant 
factor approximation is known), or that a approximation is impossible, for some c > 0. For 
minimization problems, Q(logn) inapproximability results are also common, as well as inapprox- 
imability results of the form 2^^°^")''. 

Arora and Lund |AL96 j notice this pattern in their survey paper and they classify optimization 
problems in four classes (called Class I to Class IV) according to the known inapproximability 
result. 

Of course, a priori, there is no reason why the approximability of natural optimization prob- 
lem should fall into a small number of cases, and, in fact, unless P = NP, for every (efficiently 
computable) function /(n) there are (artificial) problems that can be approximated within /(n) 
but no better. The interesting question is whether the pattern is an effect of our proof techniques, 
or whether the approximability of natural problems tends indeed to fall into one of a few typical 
cases. The latter possibilities has interesting precedents in complexity theory: for example almost 
all the natural decision problems in NP are either known to be solvable in polynomial or known 
to be NP-complete, even though, if P 7^ NP, there are (artificial) problems that have an infinite 
variety of intermediate complexities |Lad75j . Schaefer |Sch78j found in 1978 an interesting "expla- 
nation" for the latter phenomenon. He considers an infinite class of boolean satisfiability problems 
in NP, and he shows that each such problem is either in P or NP-complete. Creignou |Cre95j . 
and Khanna, Sudan, Trevisan and Williamson KSTWOOj (see also the monograph CKSOlj ) ap- 
ply the same framework to optimization problems, and definite infinite classes of minimization of 
maximization problems that can be expressed in terms of boolean constraint satisfaction. They 
show that maximization problems in the class are either solvable in polynomial time, or can be 
approximated within a constant factor but not to arbitrary good constant factors (Class I in the 
classification of |AL96j ). or are hard to approximate within a factor n'^ for some c > (Class IV), 
or it is intractable to even find a feasible solution. Minimization problems are either solvable in 
polynomial time, or fall into one of five equivalence classes (two corresponding to Class II, two 
corresponding to Class III and one corresponding to Class IV), or it is intractable to find a feasible 
solution. It is remarkable that the inifinite class of problems studied in |Cre95l IKSTWOOj would 
show only a constant number of distinct approximability behaviour, and that these cases would 
allign themselves with the classification of |AL96j . 

Recent results, however, have shown that some natural problems have approximation thesholds 
that do not fit the above scheme. 

For example, Halperin and Krauthgamer |HK03j prove a 0((logn)^~^) inapproximability re- 
sult for the Group Steiner Tree problem for every e > 0, a negative result that applies even to 
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the restriction of the problem to trees. The problem can be approximated within a 0((logn)^) 
factors in trees ICtKRODI and 0((logn)'^) in general graphs |Bar96| iFRTOSj . This result shows 
that there are natural problems for which the best possible approximation is super-logarithmic but 
polylogarithmic. 

Chuzhoy and Naor |CN04j prove an 17 (log log n) inapproximability result for the Min-congestion 
Unsplittable Flow problem. An 0(log n/ log log n)-approximate algorithm by Raghavan and 
Thompson |RT87j is known for this problem. This result shows that there are natural problems for 
which the best possible approximation is sub-logarithmic but super-constant. 

The most surprising result along these lines is the recent proof by Chuzhoy et al. |CGH"'"04] 
of a log* k — 0(1) inapproximability result for the Asymmetric fc-Center Problem. An 0(log* k)- 
approximate algorithm is known |PV98| lArcOlj . 



7.5 Inapproximability Results versus "Integrality Gaps" 

Several approximation algorithms are based on linear programming or semidefinite programming 
relaxations of the problem of interest. Typically, in order to prove that an algorithm based on a 
relaxation is r-approximate, one proves the stronger statement that the algorithm finds a solution 
whose cost is within a factor r from the optimum of the relaxation. In particular, this shows that 
the optimum of the relaxation is within a factor r from the optimum of the combinatorial problem. 

For example, the Goemans- Williamson approximation algorithm for Max CUT |(TW95j finds a 
cut whose cost is at least a = .878 . . . times the cost of the optimum of the relaxation. Such an 
analysis also proves, as a consequence, that the optimum of the relaxation is always at most 1/a 
times the true optimum. 

In order to show that such an analysis is tight, one can try and construct an instance of the 
combinatorial problem for which the ratio between the true optimum and the optimum of the 
relaxation is as large as the bound implied by the analysis of the approximation algorithm. For 



example, for Max CUT, Feige and Schechtman |FSn2j prove that there are graphs for which the ratio 
between the cost of the optimum of the relaxation and the cost of the maximum cut is arbitrarily 
close to a. 

The worst-case ratio between the true optimum of a combinatorial problem and the optimum 
of a relaxation is called the integrality gap of the relaxation. Research on integrality gaps can be 
seen as the study of inapproximability results that apply to a restricted kind of algorithms (namely, 
algorithms that use the relaxation in their analysis). Arora et al. |ABLn2j have recently shown that 
it is possible to find instances that give large integrality gaps for entire families of relaxations. It 
would be interesting to generalize their approach to semidefinite programming relaxations. 

The study of integrality gap has, moreover, a more direct, if not well understood, relation 
to inapproximability results. For example, the inapproximability result for the Set Cover problem 
proved by Lund and Yannakakis ILY94j (and its refinements and improvements in |B(TLR,93llFei98j ) 
relies on a reduction from PCP that involves a family of sets similar to the one used to prove the 
integrality gap of the standard linear programming relaxation of Set Cover. Similarly, the inap- 
proximability result for Group Steiner Tree |HK03j involves a reduction that constructs an instance 



similar to the one used in |HKK"'"n3 to prove an integrality gap. A similar progress from integrality 



gap to inapproximability result can be see in CGH"'"04 for the Asymmetric X-Center Problem 



Recently, Khot and others |KKM()fl4j present a PCP construction that shows that Max CUT can- 
not be approximate to within a factor small than a, assuming that the unique games conjecture 
holds and that the majority function has certain extremal properties. The PCP construction of 
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|KKMO04| is stronly inspired by the graphs constructed by Feige and Schechtman |FS02j to bound 
the integrahty gap of the Goemans-Wilhamson relaxation. 

Of course it would not be surprising to see that the instances created by a reduction in an 
inaproximability result have a large integrality gap: if not, we would have P = NP. What is 
surprising is that instances defined specifically to prove integrality gaps turn out to be useful as 
components in reductions proving inapproximability results. 



8 Conclusions 

In this paper we surveyed inapproximability results, a field almost non-existent in 1990, and then 
defined by the connection with probabilistic proof-checking of FGL"'"96l IAS98| IALM^98 . Progress 



in the years after the POP Theorem was extremely rapid, even outpacing the expectations of the 
researchers working in this area. For example, Bellare et al. and Arora BGS98, AroQSj presented 
two independent arguments that "current techniques" could not prove an inapproximability result 
stronger than \/n for Independent Set just months before Hastad distributed a preliminary version 
of his work |Has99j showing inapproximability within a factor n^~'=. By 1997, optimal inapprox- 
imability results were known for Max 3SAT HasOl^, Independent Set ^Has99j . Coloring FK9^ 
and Set Cover |Fei98j . the problems discussed in Johnson's paper jJoh74j that defined the study of 
approximation algorithms. 

After all these successes, the field is still very active and far from mature. Recent work |DSn2[ 
lKKMO04j has emphasized the use of higher mathematics in the analysis of PCP verifier, and of 
constructions of verifiers that are so specialised to the application to the problem of interest that 
it is hard to draw the line between PCP construction and reduction from PCP to problem. The 
connection between average-case complexity and inapproximability is likely to become a major line 
of work in the future. It is notable that several important inapproximability results have been 
announced just in the last few months, while we were writing this paper, and are still unpublished 
at the time of writing |Khon4al IKhon4bl lK'KM()n4| . 

Some important open questions seem to still be beyond the reach of current techniques, and 
one can look forward to the exciting new ideas to come in the future to address these questions. 
Here we mention some questions that are open at the time of writing and that the author feels are 
particularly interesting. Of course this is only a very small sample, and it is very biased by the 
author's interests. 



Prove Optimal Results about 2-Query PCPs. Several PCP constructions and inapproxima- 
bility results rely on versions of the PCP theorem in which the verifier reads only two entries in 
the proof, where each entry contains not just one bit but an element of a larger alphabet of size, 
say t = t{n). It is believed^'^ that there is a version of the PCP Theorem with a 2-query verifier 
that uses O(logn) randomness, accesses a proof written with over an alphabet of size t{n), and has 
soundness l/t^(n) with c > 0. Raz |R,az98j proves such a result for constant t. For general t{n), 
Raz's construction has a verifier that uses 0{t{n) ■ logn) randomness. Raz and Safra |RS97j and 
Arora and Sudan |AS97j prove a version of the result where the verifier makes a constant (larger 
than 2) number of queries, and t{n) is restricted to be n"^^-*. 

This conjecture was first made in |BGLR,93| . 
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Settle the "Unique Games Conjecture." The unique games conjecture, formulated by 
Khot |Khon2b] . is that a certain strong form of the PCP theorem holds. The conjecture implies 
the resolution of various important open questions |Kho02bl IKR03j , including the non-existence of 
(2 — e)-approximate algorithms for Vertex Cover and of (k — e)-approximate algorithm for Vertex 
Cover in A;-uniform hypergraphs. If the unique games conjecture holds, and a certain conjecture 
about extremal properties of monotone functions is true, then an optimal inapproximability result 
for Max CUT |KKMO04| would follow, which would be a remarkable breakthrough. 

Prove a Strong Inapproximability Result for Metric TSP. The Metric TSP is one of the 

most well studied optimization problems, and there is still a very large gap between the known 3/2- 
approximate algorithm |Chr76 j and the 220/219 inapproximability result |PV00j . The reduction of 
Papadimitriou and Vempala is an extremely optimized one, and it starts from the optimal version of 
the PCP Theorem of Hastad [HasOlj . so it looks like a very different approach is needed to prove, 
say, a 1.1 inapproximability result. New results for Vertex Cover |DS02j blur the line between 
PCP construction and reduction, and use PCP techniques to directly construct an instance of the 
problem of instance. It is possible that a similar approach could give stronger inapproximability 
results for Metric TSP. 

Make Progress on Graph Partitioning Problems. For many problems involving graph par- 
titioning there is a big gap between algorithms and inapproximability results. Often, no inapprox- 
imability results are known at all. The sparsest cut problem is a typical case of an important graph 
partitioning problem for which no inapproximability result is known. In the sparsest cut problem 
we are given an undirected graph G = (y,E), and the goal is to find a partition {S,S) of V that 
minimizes 

EiS,S) 
mm{\Sl\S\} ■ 

The objective function is defined in such a way that the optimal solution tends to be a cut that is 
both balanced and sparse, a property that is useful in the design of divide-and-conquer algorithms. 

The best known algorithm for Sparsest Cut is 0(T/Iogn)-approximate, where n is the number of 
vertices, and it is due to Arora, Rao and Vazirani |ARV04j . improving an O(logn) approximation 
algorithm by Leighton and Rao |LR,99j . The paper of Leighton and Rao |LR,99j describes various 
application of approximate sparsest cut computations to design divide-and-conquer algorithms. 

There is no known inapproximability result for this problem, and Khot's inapproximability 
result for the related problem Min Bisection |Khon4b] seem not to generalize to Sparsest Cut. 
Even using average-case assumptions as in |Fei02bj it is not known how to rule the existence of a 
PTAS for Sparsest Cut. 
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